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Session  Outline 


•  RT-18  Overview 

-  Motivation,  Goals  and  Tasks 

•  Critical  Review  of  Current  State  of  Knowledge 

•  Exploring  Methods  for  Valuing  Flexibility 

-  Total  Ownership  Cost 

-  Real  Options/Knowledge  Value  Added 

-  Common  Use  Case  Study  -  Modular  Munitions 

•  Gaps,  Plans  and  Priorities 

-  Gap  Analysis 

-  Plans  and  Priorities 

•  Input  from  Key  Constituents 


11/9/2010 


RT-18  SERC  ASRR  Review 


What  are  we  trying  to  do? 


•  Primary  objectives 

-  Develop  convincing  quantitative  methods,  processes,  and  tools 
(MPTs)  for  determining  the  value  of  flexibility  in  DoD  contexts 

-  Develop  associated  MPTs  for  cost-effectively  improving  flexibility  in 
DoD  contexts 

-  Identify  gaps  in  current  approaches  for  valuing,  improving  and 
incorporating  flexibility  in  the  context  of  future  DoD  needs 

•  Value-based  definition  of  "flexibility" 

-  A  system  is  flexible  to  the  extent  that  it  can  be  cost-effectively 
modified  to  meet  new  needs  or  to  capitalize  on  new  opportunities 

•  "Cost"  includes  dollars,  calendar  time,  critical  skills,  and  other  scarce 
resources  (facilities,  equipment,  supplies,  etc.).  It  also  includes  the 
costs  of  flexibility-induced  decrements  in  other  system  attributes 
(performance,  security,  safety,  usability,  etc.) 

•  "Effectiveness"  includes  improvements  in  military  outcomes  across  a 
range  of  weighted  scenarios,  and  cost  avoidance  (e.g.,  cost  of  delay) 


Basic  Issues  in  Valuing  Flexibility 
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•  If  you  do  not  need  it,  it  has  no  value 

-  Flexibility  is  needed  only  in  presence  of  uncertainty 

-  Characterizing  downstream  uncertainty  is  difficult  (yet  essential) 

*  If  you  cannot  afford  it,  you  will  not  incorporate  it 

-  Quantifying  benefits  -  and  cost  of  flexibility 

-  Estimating  current  value  of  flexibility 
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How  is  it  done  today? 


*  Several  definitions  and  methods  exist  for  valuing  flexibility 

-  Many  definitions  are  qualitative  or  descriptive 

-  Current  methods  for  valuing  flexibility  are  not  consistent  with  flexibility 
metrics,  have  unrealistic  assumptions  and  do  not  work  well  with  DoD 
measures-of-effectiveness 

•  Several  classes  of  MPTs  for  improving  flexibility 

-  Modular  and  service-oriented  architectures;  domain  ontologies; 
interoperability  connectors;  autonomy  and  adaptive  control;  agile  methods; 
concurrent  engineering;  user  programmability 

•  Little  unifying  theory 

-  Or  guidance  on  which  MPTs  work  best  in  which  situations 

*  In  practice,  it  is  generally  not  done  today 

-  Acquisition  infrastructure  optimized  around  fixed-price,  build-to-spec 

-  Temptation  in  evolutionary  development  to  go  easiest-first 

•  With  current  infrastructure,  leads  to  inflexible  initial  point  solution 

-  Flexibility  is  recognized  as  a  desirable  property  of  systems,  but  justification 
for  expending  additional  resources  to  obtain  flexibility  is  lacking 
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Tasks  proposed  in  RT-18 


*  Survey  state-of-the-art  in  valuing  flexibility 

-  Definition  of  flexibility 

-  Metrics  for  quantifying  flexibility 

-  Approaches  for  valuing  and  incorporating  flexibility 

*  Capability  to  value  flexibility  (2  or  3  approaches) 

-  To  handle  foreseeable  sources  of  change 

-  Using  monetizable  metrics,  such  as  total  cost  of  ownership 

-  Advanced  methods  for  valuing  flexibility  for  DoD  relevant  metrics 

*  Validate  MPTs  using  DoD  case  studies  (2  or  3) 

-  Completed  projects 

-  Prospective  projects 

*  Gap  analysis  of  current  MPTs  for  valuing  flexibility  and  DoD 
needs  for  future  systems 

-  Research  roadmap  6 


Current  state-of-the-art 
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Flexibility:  Concepts,  Metrics,  Approaches 


Measures 


Counting  number  of  choices 

Number  of  options  or  system 
states  affected 

Graph  theoretical  measure 

Entropy  measures 

Uncertainty  and  dispersion 
measures 

Impacts  on  other  -ilities 


Monetized  Analysis 


Total  ownership  cost 


Enablers 


Interoperability 
Modularity 
Product  platforms 
Adaptability  factor 
Standardization 
Self-organizing  systems 
Reconfigurable  components 


Real  options 

Insurance  analysis 
Risk  analysis 

Attribute  tradeoff  analysis 
Cost  of  delay  analysis 
Product  line  ROI  analysis 
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Incorporating  Flexibility  in  Systems 


Modularity 

-  Modularization  of  the  system's  architecture  around  foreseeable  sources  of  change 


Design  for  Adaptability 

-  System  adaptation  through  feedback  control  mechanisms 


Design  for  Changeability 

-  System  design  comprised  of  four  strategic  attributes:  flexibility,  agility,  robustness,  and  adaptability 


Acquisition  Strategy 

-  Cultivate  flexibility  in  the  design  process  to  some  degree  via  acquisition  strategies 


Methodology  for  Assessing  the  Adaptability  of  Products  (MAAP) 

-  Identification  of  improvement  potential  in  design  through  a  priori  applicability  requirement 
knowledge 


Open  Architecture 

-  Extension  of  software  domain  ideas  into  hardware  design  by  embedding  knowledge  and  rules 
about  possible  product  differentiations  into  products 
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Relation  with  Other  -ilities 


Flexibility 

“Ability  to  react  to 
external  changes  at 
minimal  cost  /  time 
during  development  or 
after  fielding” 


Robustness 

“Sustain  key 
capabilities 
irrespective  of 
environment” 


Adaptability 

“Embed  within 
systems  the  ability  for 
changes  at  the 
tactical  edge,  as  the 
mission  evolves  in 

Tradeoffs  unplanned  and 

unforeseen  ways” 

Performance 
Usability 
Assurance 
Quantity 
Time  to  field 

Agility 

“Change  quickly/ 
rapidly” 
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Classification  Framework 


\ 


Will  the  system  change? 

-  change  within  the  system 

-  accommodate  change 


\ 


What  is  the  source  of  change? 

-  internal 

-  external 


/ 


/ 


Is  the  change  foreseeable? 

-  known  -  knowns 

-  known  -  unknowns 

-  unknown  -  unknowns 


\ 


When  may  the  change  occur? 

-  before  deployment 

-  after  fielding 


v  y 


What  measures  of  change  efficiency  apply? 

-  quantification  of  system  change  efficiency 

-  time,  cost , ... 
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Classifying  Current  Work 


FLEXIBILITY 

System  Change? 

Measure(s)  of 
Change  Efficiency 

Source  of 
Change 

Foreseeable? 

When  Change 

Occurs 

Other 

SOURCE 

Yes 

Not 

Necessarily 

Quickly 

Cost- 

Effectively 

Int 

Ext 

Reqmnt 

Known 

Unk 

Prior  to 
Fielding 

After 

Fielding 

Related  Terms 

Thomke,  1997 

"modifying  a 
product” 

"incremental  cost 

and  time" 

X 

X 

Roser,  1999 

”change 

performance" 

"minor  time  and 

costs” 

Schulz,  1999 

"system  to  be 
changed” 

"changed  easily" 

Component  of  Changeability 

Bordoloi,  1999 

"change  states" 

Efficiency  and  Adaptability 

Olewnik,  2001 

"changes  in 
configuration” 

"real- 

time" 

X 

X 

X 

X 

Robustness  and  Adaptability  are 
modes  of  Flexibility 

Palani,  2003 

"design 

changes” 

"ease"  of  change 

Nilchiani,  2003 

"increasing  con¬ 
trol  capacity" 

X 

X 

Banerjee,  2004 

"support  new 
functions" 

Nikhiani,  2006 

"ability  to 
respond" 

"timely" 

"cost- 

effective" 

X 

X 

Qureshi,  2006 

"responsive¬ 

ness” 

"degree  of 
responsiveness" 

Keese,  2007 

"redesign" 

"quickly" 

"inexpen¬ 

sively" 

X 

X 
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State  of  Knowledge 
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•  Lack  of  consistency  in  flexibility  definitions 

•  Confused  with  other  "-ilities" 

•  Lack  of  connection  between  flexibility  and  value 

•  Lack  of  relationship  between  flexibility  and  design 
variables 


"\ 


J 

"\ 


J 


J 


J 


(  I  ^ 

•  Lack  of  "scholarly  maturity" 

"concept  of  flexibility  is  today  where  the  concept  of  quality  was  20 
years  ago"  [Saleh  2009] 
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Exploring  Methods  for  Valuing 
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Total  Ownership  Cost  (TOC)  Approach 


*  TOC  Advantages,  Challenges,  Strategies 

-  Representative  examples 

*  TOC  Analysis  for  Foreseeable  Change 

-  Model  and  tool  for  individual  systems 

*  Calibrated  to  TRW  C4ISR  software  data  (3  systems) 

*  Exploring  calibration  to  NPS  SHIPMAIN  hardware  data 

-  Model  and  tool  for  families  of  systems 

*  Calibrated  to  COCOMO  II  software  data  (161  projects) 

*  Exploring  calibration  to  AFIT  modular  munitions  hardware  data 

*  Candidate  Extensions 

-  Refined  and  extended  model  capabilities 

*  Particular  domains,  tradeoff  analyses,  enterprise  analysis 

*  Effects  of  adaptation  to  unforeseeable  change 

-  Integration  with  alternative  valuation  models 

10/27/2010  Valuing  Flexibility  via  TOC  15 


SYSTEMS  ENGINEERING 

Research  Center 


TOC  Advantages,  Challenges,  Strategies 


*  TOC  Advantages 

-  Increasingly  required  (DoDI  5000.02,  WSARA  2009) 

-  Easy  to  understand  across  specialty  domains 

-  Clear  cause-effect  relationships,  straightforward  calibration 

*  TOC  Challenges 

-  Defining  flexibility  investment  costs,  resulting  cost  reductions 

*  Rework  and  change-adaptation  cost  reductions  a  proxy  for  benefits 

-  Predicting  uncertain  futures 

*  TOC  Approach  Strategies 

-  Tailor  analysis  approaches  to  common  situations 

*  DoDI  5000.02  milestone  reviews,  make-or-buy  decisions 

-  Explicitly  emphasize  need  to  define  evolution  requirements 

*  Not  just  snapshot  capability,  interface,  KPP,  project  requirements 

-  Start  with  simple  models  and  tools,  refine  and  extend  as  needed 
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%  of  Cost  to  Fix  SPR’s 


Point-Solution  Architectures  Cause  Major  Rework 

C4ISR  Contracts:  Nominal-case  requirements;  90  days  to  PDR 


0  10  20  30  40  50  60  70  80  90  100 


%  of  Software  Problem  Reports  (SPR’s) 
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C4ISR  Project  C:  Architecting  for  Change 

USAF/ESC-TRW  CCPDS-R  Project* 


50  -i 


40 


30  - 


O 

O 

c/) 

'cn 


3 

O 

I 

CD 

j?  20 

I 


10  - 


-►  Design  changes:  Architecture  changes  that 
typically  span  multiple  components  and  teams 


Implementation  changes:  Pre-FQT 
changes  that  are  typically  isolated  to  a 
single  component  and  team 


Maintenance  changes:  Include  some 
out-of-scope  changes  performed 
under  separate  contract 


Maintenance 

Changes 


PDR 

A 


14 


CDR 

▲ 

24 


FQT 

A 

48 


Common  Subsystem  Schedule  (months) 
Figure  D-14.  Common  Subsystem  adaptability 


When  investments  made  in  architecture, 
average  time  for  change  order  becomes 
relatively  stable  over  time... 

*  Walker  Royce,  Software  Project  Management:  A  Unified  Framework.  Addison-Wesley,  1998. 
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Single-System  TOC  Model  Example 


A 

B 

C 

D 

E 

1 

Input  Parameters 

System 

2 

A 

B 

C 

3 

Software  Size  (KSLOC) 

100 

100 

355 

4 

#  Change  Requests/Release 

373 

1005 

1600 

5 

#  Change  Requests  (l&T  only) 

6 

#I&T  Change  Requests/Release/  >1  PM 

27 

22 

7 

#Total  Change  Requests/Release/  >  1  PM 

16 

8 

Change  Request  Fix  Time  (See  assumption  #2) 

261 

356 

263 

9 

Total  Effort  (Person  Months) 

731 

865 

1900 

10 

%  Arch,  RESL 

- 5% 

5% 

25% 

11 

%  Rework,  RVOL  ^ 

41.16% 

13.85% 

12 

13 

Cumulative  Total  Cost  of  Ownership 

Project  A 

Project  B 

Project  C 

14 

Cycle  1 

40.70% 

46.16% 

38.85% 

15 

Cycle  2 

76.41% 

87.31% 

52.70% 

16 

Cycle  3 

112.11% 

128.47% 

66.55% 

17 

Cycle  4 

147.82% 

169.62% 

80.40% 

18 

Cycle  5 

183.52% 

210.78% 

94.25% 
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Relative*  Total  Ownership  Cost  (TOC) 


Project  A  - Project  B  - Project  C 


*  Cumulative  architecting  and  rework  effort  relative  to  initial  development  effort 
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Example  of  Use:  MS  A  Preparation  Guidance 


•  Alternatives  analyzed  shall  include  at  least  one  architecture 
organized  around: 

-  Common  sources  of  life-cycle  change 

•  For  RPVs,  these  usually  include  user  interface  displays  and 
controls,  new  payloads,  self-defense  and  electronic  warfare,  data 
fusion,  NCSoS  protocols,  and  hardware  maintainability 

-  Risk  analysis  and  prototyping  of  critical  off-nominal  scenarios 

•  For  RPVs,  these  usually  include  communications  outages,  anti-RPV 
threats,  noisy  and  intermittent  data  sources,  redirected  missions, 
and  cross-RPV  coordination  of  responses 

•  Analyses  of  alternatives  shall  include  total  ownership  cost 
comparisons 

-  Based  on  relevantly-calibrated  life  cycle  cost  models 


10/27/2010 


Valuing  Flexibility  via  TOC 


21 


Systems  Product  Line  Flexibility 

Value  Model 


For  Set  of  Products: 

•  Average  Product 
Cost 

•  Annual  Change 
Cost 

•  Ownership  Time 

•  Percent  Mission- 
Unique,  Adapted, 
Reused 

•  Relative  Cost  of 
Developing  for  PL 
Flexibility  via  Reuse 

•  Relative  Costs  of 
Reuse 


Systems 
^1  PL  Flexibility 
™  Value  Model 


I 


4 


As  Functions  of  # 
Products,  #  Years  in 
I  Life  Cycle: 

I*  PL  Total 

Ownership  Costs 
I  •  PL  Flexibility 
I  Investment 
I  •  PL  Savings  (ROI) 
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Sensitivity  Analysis  Example 


Valuing  Flexibility  via  TOC 
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Conclusions  and  Candidate  Extensions 


*  TOC  approach  has  several  advantages 

-  Increasingly  required  (DoDI  5000.02,  WSARA  2009) 

-  Easy  to  understand  across  specialty  domains 

-  Clear  cause-effect  relationships,  straightforward  calibration 

*  Important  to  determine  evolution  requirements 

*  Basic  models  available  for  foreseeable  change 

-  Can  be  used  to  support  TOC-based  milestone  decisions 

-  Individual  systems,  families  of  systems 

-  Best  to  have  calibration  data 

*  Candidate  Extensions 

-  Refined  and  extended  model  capabilities 

*  Particular  domains,  tradeoff  analyses,  enterprise  analysis 

-  Integration  with  alternative  valuation  models 
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sY?e“^rc^^f':iNG  Projects  A  and  B  Major  Rework  Sources 


-  Change  processing  over  1  person-month  =  152  person-hours 


Category 

Project  A 

Project  B 

Extra  long  messages 

3404+626+443+328+244=  5045 

Network  failover 

2050+470+360+160=  3040 

Hardware-software  interface 

620+200=  820 

1629+513+289+232+166=  2832 

Encryption  algorithms 

1247+368=  1615 

Subcontractor  interface 

1100+760+200=  2060 

GUI  revision 

980+730+420+240+180  =2550 

Data  compression  algorithm 

910 

External  applications  interface 

770+330+200+160=  1460 

COTS  upgrades 

540+380+190=  1110 

741+302+221+197=  1461 

Database  restructure 

690+480+310+210+170=  1860 

Routing  algorithms 

494+198=  692 

Diagnostic  aids 

360 

477+318+184=  979 

TOTAL: 

13620 

13531 
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Real  Options/Knowledge  Value  Added 


KVA+RO+IRM  are  a  combination  of  method  and 
toolsets  to  assist  the  executive  in  decision  making 


•  Knowledge  Value  Added  (KVA)  is  a  method  that  systematically  expresses  non¬ 
revenue  activities  in  common  units  of  output  to  quantify  value 

•  Real  Options  (RO)  provides  a  way  to  qualitatively  and  quantitatively  evaluate  the 
relative  value  of  various  courses  of  action  under  consideration 

•  Integrated  Risk  Management  combines  KVA  and  RO  with  a  powerful  toolset  to 
assist  the  program  manager  in  the  decision  process 

•  Treatment  of  parameters  as  distributions  permits  rigorous  analysis  in  an  uncertain 
world,  where  instead  of  single  point  estimates,  we  use  ranges  as  inputs 

•  Monte  Carlo  risk  simulation  and  process  models  permit  consideration  of  possible 
outcomes  within  a  reasonable  time  period 

•  Disciplined  processes  yield  defensible  results  that  can  be  updated  as  more 
knowledge/  information  is  realized  by  the  program 

•  Risk  simulation,  sensitivity  analysis,  and  forecasting  are  automated  (analyses  are 
efficient,  quick,  consistent,  replicable,  defensible,  and  scalable) 


Measuring  Value 

KVA  quantifies  the  value  of  the  knowledge  used  to  produce 
common  units  of  output 

Shows  decision  makers  the  benefit  and  cost  of  each  program  or 
project 

Measures  how  resources  are  allocated  in  terms  of  the  volatility 
of  productivity  (e.g.,  ROI) 

Providing  ROI  volatility  inputs  to  IRM 

Military  value  in  this  study  was  postulated  to  be  represented 
by  capability  provided  to  the  warfighter  measured  in  a  variety 
of  ways 

-  Strategic  importance  as  represented  by  OPNAV  sponsor  priorities 

-  Technical  value  as  represented  by  acquisition  community  priority 

-  Functional  complexity  represented  by  Delivered  Source  Lines  of  Code  (DSLOC) 

-  Subject  Matter  Expert  evaluation  of  complexity  and  mission  criticality, 
aggregated  from  the  component  level 


RISK  MITIGATION  d  j  RISK  IDENTIFICATION 


Integrated  Risk  Management(IRM) 


OList  of  projects 
and  strategies 


OBase  case  projections 
for  each  project 


O  Develop  static 
financial  models 


O  Dynamic  Monte 
Carlo  simulation 


Z 

O 


Time  Series  Forecasting 


0 

z 

3 

a 

c 

S 


Start  with  a  list  of  projects 
or  strategies  to  be 
evaluated...  these  projects 
have  already  been  through 
qualitative  screening 


Framing 
Real  Options 


with  the  assistance  of 
time-series  forecasting, 
future  outcomes  can  be 
predicted... 


Options  analytics, 
simulation,  optimization 
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RISK 
SIMULATION 


Lognoimal 


the  user  generates  a 
traditional  series  of  static  base 
case  financial  (discounted  cash 
flow)  models  for  each  project. . 


Portfolio  optimization 
and  asset  allocation 


..  Monte  Carlo  simulation  is  added 
to  the  analysis  and  the  financial 
model  outputs  become  inputs  into 
the  real  options  analysis.. 


O  Reports  presentation 
and  update  analysis 


...the  relevant  projects 
are  chosen  for  real 
options  analysis  and  the 
project  or  portfolio  real 
options  are  framed.. 


Simulation  Lattice 


...real  options  analytics  are 
calculated  through  binomial  lattices 
and  closed-form  partial-differential 
models  with  simulation... 


> 

~  ...  stochastic  optimization  is  the 

^  next  optional  step  if  multiple 


vi  projects  exist  that  require  efficient 
2  asset  allocation  given  some 
budgetary  constraints...  useful  for 
strategic  portfolio  management. . . 
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...  create  reports,  make 
decisions,  and  do  it  all 
again  iteratively  overtime. 


The  IRM  toolset  provides  powerful  tools  for  analyzing 
the  data 

•  Treatment  of  parameters  as  distributions  permits  rigorous 
analysis  in  an  uncertain  world 

•  Instead  of  single  point  estimates,  we  use  ranges  as  inputs 

•  Monte  Carlo  simulation  and  process  models  permit 
consideration  of  all  possible  outcomes  within  a  reasonable 
time  period 

•  Disciplined  processes  yield  defensible  results  that  can  be 
updated  as  more  knowledge/  information  is  realized  by  the 
program 

•  Risk  simulation,  sensitivity  analysis,  and  forecasting  are 
automated  (analyses  are  efficient,  consistent,  replicable, 
defensible,  and  scalable) 


AEGIS  Case:  Introduction  and  Context 


Introduction  of  Open  Architecture  (OA)  business  and 
technical  processes  provides  opportunity  to  improve 
acquisition 

-  Increased  competition 

-  Shorter  cycle  time 

-  Reduced  total  ownership  and  acquisition  cost 

The  AEGIS  Advanced  Capability  Build  (ACB)  process  is  one 
implementation  of  the  OA  approach 


The  ACB  Process 

The  ACB  process  provides  for  software  updates  to  ships 
within  the  program  on  a  two-year  cycle 

ACBs  are  identified  by  the  first  year  in  which  they  will  be 
fielded,  e.g.,  ACB-14 

US  Navy  CGs  and  DDGs  will  be  inducted  into  the 
process  as  they  receive  computing  plant  updates 
during  major  availabilities  that  convert  the  processors 
and  networks  to  a  COTS-based  configuration 

The  hardware  baseline  that  supports  OA  must  be  in 
place  to  begin  execution  of  the  ACB  process 

Once  a  ship  is  inducted,  it  will  receive  the  scheduled 
update  plus  any  previous  updates  (e.g.,  ACB  16  ships 
entering  the  program  will  receive  ACB  14  capabilities 
as  well) 


The  problem  addressed  in  this  study  deals 

with  risk  and  value 

•  Value  is  realized  through  fielding  of  military  capability  for  the  war¬ 
fighter 

•  Risk  is  found  in  uncertainty 

-  Cost  uncertainty  creates  budget  risk 

-  Technology  risk  can  lead  to  schedule  and  budget  risk 

•  This  study  provided  a  pilot  implementation  of  the  Knowledge  Value 
Added  +  Integrated  Risk  Management  method  to  represent  value  and 
risk  to  assist  the  PM  and  sponsor  in  selecting  the  proper  capability  mix 
to  field  in  a  given  ACB 

•  The  problem  space  considered  23  capabilities  to  be  implemented 
through  changes  to  32  software  components  across  three  scheduled 
ACBs  (ACB  14, 16,  and  18) 

•  Given  the  universe  of  desired  capabilities,  the  problem  is  to  select 
those  providing  the  best  value  to  the  war-fighter  for  inclusion  in  a 
given  ACB  subject  to  budget  constraints,  risk  and  uncertainty  of  cost 
and  timing 


The  study  articulated  a  notional  value  of  military 
value  and  used  powerful  financial  and  analytical 

tools 


•  Knowledge  Value  Added  (KVA)  provides  ways  of 
representing  outputs  (value)  in  common  units 

•  Real  Options  provides  tools  to  compare  the  value  stream 
of  various  options  in  rigorous  terms 

•  Integrated  Risk  Management  considers  uncertainties  and 
represents  risk  in  quantitative,  clear  and  defensible  terms 


Model  input  assumptions  areComTnsizi"ginputsaJndulsingwei1ghtsto 

1  1  obtain  the  expected  military  value 


entered  on  a  data  sheet  ^ 


I 


I  I 


Common  Size  Factor 


Capability 

Capability  1 

Capability  2 

Capability  3 

Capability  4 

Capability  5 

Capability  6 

Capability  7 

Capability  8 

Capability  9 

Capability  10 

Capability  11 

Capability  12 

Capability  13 

Capability  14 

Capability  15 

Capability  16 

Capability  17 

Capability  18 

Capability  19 

Capability  20 

Capability  21 

Capability  22 

Capability  23 

Starting  with  23  capabilities 
(more  to  be  added  later  when 
there  is  sufficient  data) 


100 

010 

200 

200 

35% 

35% 

15% 

15% 

SMEVAand 
DSLOC  Only 

AJI  Weighted 
A  Common 
Sized 

OPNAV  & 
Technical 
Pnonty 

SUE  Mean 
alue  Added 

TechrecaJ 

Pnonty 

OPNAV 

Pnonty 

DSLOC 

Complexity 

Cost 

S*mAatK>n 

Technical 
Pnonty  K-L 

OPNAV 
Pnonty  H- 

EMV  Score 

EMV  Score 

EMV  Score 

1 

Objective 

Used 

CV 

43  00 

3 

2 

278 

29  00 

21 

22 

32100 

37  68 

43  00 

43  00 

0  1834 

?8  00 

2 

1 

541 

126  00 

22 

23 

569  00 

42  24 

45  00 

45  00 

0  0903 

13  00 

1 

3 

58 

77  00 

23 

21 

7100 

19  78 

44  00 

44  00 

0  0719 

40  33 

5 

4 

635 

2100 

19 

20 

675  33 

48  04 

39  00 

39  00 

0  0576 

1767 

7 

7 

134 

15  00 

17 

17 

15167 

21  07 

34  00 

34  00 

0  0407 

35  67 

4 

6 

392 

27  00 

20 

18 

427  67 

37  60 

38  00 

38  00 

0  1568 

50  67 

6 

5 

549 

17  00 

18 

19 

599  67 

48  05 

37  00 

37  00 

0  0481 

47  67 

10 

11 

675 

77  00 

14 

13 

722  67 

48  41 

27  00 

27  00 

0  0689 

19  00 

8 

8 

109 

16  00 

16 

16 

128  00 

20  07 

32  00 

32  00 

0  0382 

33  67 

11 

9 

189 

10  00 

13 

15 

222  67 

26  80 

28  00 

28  00 

0  0603 

20  00 

9 

10 

88 

300 

16 

14 

108  00 

18  78 

29  00 

29  00 

0  0707 

27  67 

14 

12 

159 

2100 

10 

12 

186  67 

2185 

22  00 

22  00 

0  1137 

27  67 

12 

13 

159 

1100 

12 

11 

186  67 

2215 

23  00 

23  00 

0  0408 

44  67 

13 

14 

523 

27  00 

11 

10 

567  67 

40  24 

2100 

2100 

0  0956 

17  67 

20 

15 

134 

10  00 

4 

9 

151  67 

14  77 

13  00 

13  00 

0  0603 

17  67 

21 

16 

134 

500 

3 

8 

151  67 

14  17 

11  OO 

1100 

0  0816 

34  33 

19 

17 

328 

900 

5 

7 

362  33 

27  10 

12  00 

12  00 

0  0454 

17  67 

22 

18 

134 

600 

2 

6 

15167 

1327 

800 

800 

0  0372 

1400 

23 

19 

81 

18  00 

1 

5 

95  00 

954 

600 

600 

0  0  340 

1767 

15 

20 

134 

14  00 

9 

4 

151  67 

14  77 

13  00 

13  00 

0  0164 

22  67 

18 

21 

144 

18  00 

6 

3 

166  67 

15  67 

900 

900 

0  0454 

11  67 

16 

22 

180 

78  00 

8 

2 

191  67 

13  38 

10  OO 

10  00 

0  0468 

10  00 

17 

23 

95 

1000 

7 

1 

105  00 

923 

800 

800 

0  0603 

Technical  and  OPNAV 
priorities 


Intermediate  computations:  risk- 
simulation  assumption,  readjusted 
priorities,  expected  military  score  and 
cost-based  risk  coefficients 


High,  most  likely,  low  cost 
estimates  for  running 
thousands  of  simulation  trials 


DSLOC  provides  a  measure  of 
complexity 


Running  the  model  provides  recommended  selections 

ACB  14  sample  results  with  $150M  budget  constraint 


Expected  Military  Value:  SME  Mean  Value  Added.  DSIOC  Complexity.  Common  Sized.  Weighted  OPNAVfTechnical  Priorities 


Risk  $ 
S 6.91 
S3. 81 
SI. 42 

52.77 
SO.  86 

55.90 
S2.31 
S3  34 

50. 77 
SI  62 
Si. 33 
S2.48 

50.90 
S3. 8  5 
$0.89- 

S1.16 
SI  23 
SO  49 
SO.  32 
SO.  24 
SO.  71 
SO.  63 
SO.  56 

S6  18 

< - 


Risk  % 


Selection 


Selection  of  EMV 
calculation  method 


18.34% 

9.03 % 

7.19% 

5.76% 

4.07% 

15.68% 

4.81% 

6.89% 

3.82% 

6.03% 

7.07% 

11.37% 

4.08% 

9.56% 

0.UJ79 

8.16% 

4.54% 

3.72 % 

3.40% 

1.64 % 

4.54% 

4.68% 

6.03% 

0.0000 

0.0000 

0.0000 

1.0000 

1.0000 

0.0000 

1.0000 

0.0000 

1.0000 

1.0000 

1.0000 

0.0000 

1.0000 

1.0000 

I  .K/UUU 

1.0000 

1.0000 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

0.0000 

Go  or  No-Go  decisions 
in  the  portfolio 
selection 


_Benefits  (EMV),  Cost, 
Risk  are  considered 


Constraints  can  be  set  (budget, 
capability  count,  FTE,  priorities,  etc) 


Starting  with  23  capabilities  (more  to  be  added 
later  when  there  is  sufficient  data) 


SHIPYARD  PLANNING  PROCESS  (MODERNIZATION  AND  MAINTENANCE) 

The  DoD  spends  more  than  $59  billion/year  on  a  broad  range  of 
defense  maintenance  capabilities  and  programs. 

Current  inventory  of  300  ships,  15,000  aircraft,  900  strategic  missiles, 
and  330,000  ground  vehicles,  the  need  for  maintenance  programs  is 
evident. 

Navy  Fleet  maintenance  and  modernization  efforts  for  fiscal  year  2005 
amounted  to  85  ship  and  submarine  scheduled  availabilities  cost  $3.9 
billion  (Hugel,  2005). 

The  Navy  must  be  extremely  diligent  in  its  maintenance  efforts.  Ships 
and  submarines  provide  great  value  to  national  defense  objectives. 
Maintenance  and  modernization  policy  is  carefully  designed  to  keep 
Navy  ships  operating  at  the  maximum  level  of  material  readiness 
possible  (OPNAVINST  4700.7K). 

This  project  looked  at  the  business  process  for  conducting  industrial 
work  in  Naval  Shipyards  and  the  effect  that  deploying  an  imaging 
technology  could  have 


3D  Scanning  and  PLM  Technology  Options 


AS-IS  situation.  Proceed  with  the 
process  without  any  attempt  to 
introduce  new  technologies.  Baseline 
situation  option  of  leaving  things  the 
way  they  are. 


Maintain  baseline 
condition 


We  can  quantify  real  options  analysis  values/results 


Maturity  (Years) 

5 

Risk-Free  Rate  (%) 

5.00% 

Strategic  Option  Valuation 

AS-IS 

TO-BE 

RADICAL 

Benefits 

$  49,175,536.83 

$ 

93,344,192.00 

$ 

95,097,452.00 

Costs 

$  44,705,033.48 

$ 

7,854,206.09 

$ 

4,488,887.70 

Volatility 

N/A 

8.04% 

9.81% 

Total  Strategic  Value 

$  4,470,503.35 

$ 

87,227,330.00 

$ 

91,601,502.00 

Factor  Increase 

19.51 

20.49 

Expansion  Valuation  on  Stage-Gate  Options 

Maturity  (Years) 

10 

10 

10 

Factor  Increase 

3 

3 

10 

AS-IS 

TO-BE 

RADICAL 

Benefits 

$  147,526,610.48 

$ 

280,032,576.00 

$ 

950,974,520.00 

Costs 

$  134,115,100.43 

$ 

23,562,618.26 

$ 

44,888,876.96 

Volatility 

N/A 

25.43% 

31.02% 

Long  Term  Total  Strategic  Value 

$  13,411,510.04 

$  265,742,275.00 

$ 

923,752,800.00 

Factor  Increase 

19.81 

68.88 

Common  Use  Case  Study 
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Initial  Case  Study  Modular  Munitions 


•  DoD  uses  modular 
munitions  to  achieve 
operational  flexibility 

•  Various  configurations  are 
allowable  based  on 
warhead,  fuze  and 
guidance  module 
selection 

•  Munitions  are  assembled 
in  the  field  according  to 
the  days  Air  Tasking  Order 


TitK’ttf ,  BLU-11C  mk-82  ,  BLV-111 


PAVEWAY 


GBU  24 


GBU-27 


GBU  28 


PAVEWAY 


GBU  10 


GBU-24 


GBU  15 
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Relating  Functionality  to  Capability 


•  Candidate  tasks  from  UJTL 

OP  3.2  Attack  Operational  Targets 
OP  3.2.1  Provide  Close  Air  Support  Integration  for  Surface  Forces 
OP  3.2.2  Conduct  Non-Lethal  Attack 
OP  3.2.4  Suppress  Enemy  Air  Defenses 
OP  3.2.5  Interdict  Operational  Forces/Targets 
OP  3.2. 5.2  Conduct  Surface/Subsurface  Firepower  Interdiction  of  Operational  Forces/ 
Targets 

OP  3.2.6  Provide  Firepower  in  Support  of  Operational  Maneuver 

•  Additional  task  definition  is  necessary 

OP  3.X.X  Defeat  fixed  surface  targets 
OP  3.X.X  Defeat  mobile  surface  targets 
OP  3.X.X  Defeat  sub-surface  targets 
OP  3.X.X  Defeat  Area  Targets 
OP  3.X.X  Defeat  Chem/Bio  Facilities 

OP  3.X.X  Limit  Collateral  Damage  (possibly  an  attribute/measure  of  other  tasks) 

OP  3.X.X  Survive  enemy  air  defenses 


Existing  and/or  proposed  system  modifications  can  be  measured 
against  the  capabilities  defined  by  tasks  such  as  these 


sysJ™Ji™»T.n™NG  Relating  System  Modifications  to  Expanded  Capability 


•  "Bunker  Buster"  of  Desert  Storm  (evolved  into  GBU-28) 

—  New  warhead  "module"  used  in  conjunction  with  existing  guidance  and  fuze 
modules 

-  Expanded  capability  for  tasks: 

OP  3.2.5.2  Conduct  Surface/Subsurface  Firepower  Interdiction  of  Operational  Forces/ 
Targets 

OP  3.X.X  Defeat  sub-surface  targets 

•  "Agent  Defeat"  munition  concept 

-  New  warhead  module  -  can  be  substituted  for  BLU-109 

-  Expanded  capability  for  task: 

OP  3.X.X  Defeat  Chem/Bio  Facilities 

•  JDAM  provided  a  new  guidance  option  for  existing  warheads  and  fuzes 

-  Expanded  capability  for  task: 

OP  3.X.X  Survive  enemy  air  defenses 

-  Expanded  capability  under  degraded  conditions  for  other  tasks  addressed  by  LGB 

•  Projected  cost  of  achieving  quantifiable  additional  capability  can  be  assessed 

-  At  the  "family  of  munitions"  level,  cost-benefit  of  modular  design  can  be  assessed 
based  on  both  operational  and  programmatic  flexibility 


Flexibility  in  Design  Feature  Space 

Assessing  a  Vector  Modularity  Measure*  (VMM) 


Provide 


*Stryker,  Jacques  and  Long;  to  appear  in  Journal  of  Engineering  Design 


•  •  • 


Research  Methodology 

Vector  Modularity  Measure  (VMM)  Defined 


0-1  0-1  0-1* * 


V  X  Y  Z 


•  Degree  of  Coupling.  V 

•  Ratio  of  entries  with 
an  "x"  to  total  entries 
across  all  five  DSMs 
minus  the  diagonal 
entries 

•  0-1 
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9 

Element  10 

X 

10 

•  Reusability,  X 

•  Ratio  of  modules  used 
in  multiple  products 
(nmp)  to  total  number 
of  modules  in  the 
product  being 
analyzed  (n) 

•  0-1 


•  Extensibility,  Z 

•  Ratio  of  architectural 
options  (a)  to  the  total 
number  of  functions  (m) 
performed  by  the 
product  being  analyzed 

•  Assumption:  a  product 
will  not  be  fielded  with 
less  than  50%  anticipated 
functionality 

•  0  -1*  (practical  limit) 
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•  •  • 


Research  Methodology 

Reconfigurability  Measure  Defined 


Reconfigurability,  Y 

n 

Number  of  modules 
in  a  product 

r 

Number  of  possible 
reconfigurations 

t 

Number  of  modules 
with  options 

S 

Total  number  of 
module  options 

ru.b. 

Upper  bound  number 
of  reconfigurations 
for  a  given  5  and  t  pair 

given  5  and  t  pair 


0  -  ru.b./5  0  -  rah!t 


0-  1 


0-1 


t 

yi  =  — 

n 


Number  of 
configurations 
per  option  in 
inventory 


Number  of 
configurations 
per  decision 
point 


How  much  of 
my  product  is 
reconfigurable? 


For  a  given  S 
and  t  pair,  is  the 
number  of 
configurations 
maximized? 


46 


Relating  Modularity  to  Flexibility* 

Modularity  vs.  A&CO 


*  Associating  degree  of  coupling  and  assembly  &  checkout 


-  Identify  assembly  and  checkout  procedures 

-  Associate  clock  times  with  each  step 

-  Identify  required  modules  to  perform  each  step  and  associated 
interface  types 

-  Summarize  clock  times  associated  with  handling  each  module  and 
associated  interface  types 


PnPSat  Assembly.  Integration,  &  Test  Flow 


i 
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*  ‘SET  - 
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(hours) 

^mbly  of  the  bus  structure  in  a  “flat-satellite"  config- 

1.37“ 

2 

Install  the  internal  romponents  and  mate  harnesses  to 
the  power  and  data  network 

1.37“ 

3 

Power  up  the  vehicle  on  internal  power  and  ran  a  bus 
functional  test  to  verify  internal  devices 

0.83 

4 

Fold  up  the  bus  panels  and  install  external  components. 

including  solar  array 

1.48 

5 

Install  payload  items 

35* 

6 

Connect  the  RF  links  from  the  vehicle  to  the  ground 

0.25 

7 

Power-up  the  vehicle  with  solar  array  simulator  power 
and  run  bus  functional  test 

1.28 

Lift  and  measure  vehicle  mass  and  center  of  gravity 

0.98 

9 

Place  on  vibration  table  and  ran  sine  sweeps  and  single 
axis  random  vibration  (Optional) 

1.17* 

10 

Power-up  and  run  bus  functional  test 

0.83* 

Demonstrate  S-Rand  and  UHF  Compatibility  to  ground 
station  simulator 

0.33 

12 

Demonstrate  array  first-motion  deployment 

0.08 

13 

Deploy  array  and  illuminate  to  verify  end-to-end  power 
flow 

0.17 

14 

Load  Mission  Flight  Software 

0.25 

15 

Run  nominal  Day  iu  the  Life  scenario  and  verify  L/EO 

3.0* 

16 

Perform  nominal  ground  commanding  and  ops  testing 
with  RF  links 

S.(T 

Perform  stressing  cases  or  off-nominal  events  (device 
POR,  system  resets) 

3.0* 

18 

Set  vehicle  for  launch 

1.5* 
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*  Oyama,  Jacques,  Stryker,  Long;  to  appear  in  Systems  Research  Forum 
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Relating  Modularity  to  Flexibility 

Simulation  Model 


Developed  in 
Arena™ 

Two  basic 
models 
(GBU-24, 
GBU-31) 

Input  data 
collected  from 
Air  Force  SMEs 

Modified 
models  to  test 
effects  of 
alternate 
designs  / 
architectures 
on  assembly 
and  checkout 
times 


1.  Bomb  Body  Preparation 

2.  Fuze  Prep  aration 

3.  Forward  Adap  ter  Queuing 

4.  Fin  Queuing 

5.  Forward  Adap  ter  Installation 

6.  Fuze  Installation 

7 .  Fin  Installation 

8.  Fully  Assembled 

9.  Final Insp  ection 


GBU-24  Assembly  &  Checkout  Model 


Other  Case  Studies 

PnPSat 


Used  to  evaluate  current  design 
and  make  recommendations  to 
further  increase  modularity 

Modularity  benefits  being  realized 
the  most:  Reusability  and 
Extensibility 


Close  to  maximizing  number  of 
reconfigurations  possible  for  5  and 
t  pair 
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*  Recommendations: 

•  Increase  options  for  the  spacecraft  bus 
module 

•  Add  "scars"  for  a  future  propulsion 
capability  (function) 

•  Determine  options  for  each  module; 
module  options  used  in  the  assessment 
are  "planned"  modules  and  are  not  yet  in 
inventory 


Gaps,  Plans  and  Priorities 
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Gap:  What's  out  there?  Will  it  work,  here? 


The  open  literature: 

•  Almost  all  treatment  of  flexibility  appears  in  the 
Engineering  Design  literature  and  Software  Engineering 
literature. 

•  There  is  no  widely  accepted  definition  of  flexibility  ... 
often  synonymous  with  other  "...ilities." 

•  Almost  none  of  this  literature  directly  addresses  the 
value  of  flexibility. 

Observation:  There  exist  no  general  off-the-shelf  modeling/ 
analysis  for  evaluating  the  value  of  flexibility. 
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Gap:  What's  out  there?  Will  it  work,  here? 


Generic  modeling/analysis: 


•  Simulation. 

•  Net  Present  Value  Analysis  (e.g.,  Total  Cost  of 
Ownership). 

•  Portfolio  Theory  (e.g..  Real  Options). 

Observation: 

Beyond  their  respective  particular  assumptions,  each  of  these  generic 
methods  shares  a  common  deficiency  in  more  general  settings. 
Because  a  high  granularity  probability  law  is  required,  none  of  the 
available  current  methods/analyses  is  immediately  extendable  to 
decision  making  driven  by  valuation  of  flexibility. 


sv^,srffissiNo  Gap:  What's  out  there?  Will  it  work,  here? 


Summary: 

•  There  is  no  useful  definition  of  flexibility,  and  hence,  no 
reported  approach  for  capturing  the  uncertainty  on  value 
of  flexibility. 

•  There  are  no  off-the-shelf  results  for  capturing  uncertainty 
in  the  value  of  flexibility. 

•  There  are  no  generic  analytical  methodologies  that 

promise  to  be  easily  adapted  to  addressing  decisions  that 
consider  the  uncertain  value  of  flexibility. 

•  There  exists  no  empirical  platform  to  exercise  new/existing 
decision  strategies  that  consider  the  uncertain  value  of  flexibility. 


Observation:  We  fully  understand  the  deficiencies  in  the 
state-of-the-art  with  regarding  to  valuing  flexibility. 
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Steps  in  advancing  the  state-of-the-art? 


Summary: 

•  Advance  an  appropriate  definition  of  flexibility. 

•  Develop  an  appropriate  analytical  framework  in  which 
decisions  directed  by  value  of  flexibility  can  be  addressed 

•  Develop  an  computational  platform  on  which  the  usefulness 
of  new  modeling/analysis  strategies  can  be  tested.  (Substantial 
effort,  low  risk) 

•  Develop  practical  new  modeling/decision  strategy  that 
overcomes  the  curse  of  required  high-granularity  probability 
laws.  (Substantial  effort,  higher  risk) 

•  Develop  decision  support  tools  that  implement  realistic/ 
practical  decision  support  that  values  flexibility. 


svsssss?s25svMO  How  can  we  advance  the  state-of-the-art? 


Predictive  modeling  and  decision  support: 

•  Accept  the  fact  that  in  practice  we  will  NEVER  capture  all 
finite  joint  probability  distributions  that  govern  the  value  of 
flexibility  (i.e.,  complete  probability  law  will  be  unavailable). 

•  Exploit  the  fact  that  not  all  uncertainty  need  be  understood 
in  order  to  identify  optimal  decisions  that  include  the 
uncertain  value  of  flexibility. 

•  Advance  a  deeper  understanding  of  the  relationship  between 
design  decisions  and  the  value  of  information. 

•  Develop  decision  support  that  focuses  on  identifying  the 
relative  preference  for  flexibility  using  attainable  (as  opposed 
to  desirable)  predictive  modeling. 
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Flexibility  Valuation  Tool 


Decision  support  for 
program  development 
and  management 
Help  acquisition 
managers  make  best 
decision  about  the 
introduction  of  flexibility 

Rapid  what/ifs 

Execute  sensitivity  with 
respect  to  information 
(information  economics) 

Manage  acquisition  of 
information  in 
procurement  process 


-  Each  decision  epoch  uses  the  best  maximum-entropy  estimate  of  risk 
associated  with  each  alternatives 

-  Provides  a  measure  of  sensitivity  to  change  in  our  underlying  alternative  risks 
with  the  acquisition  of  additional  (perhaps  costly)  information 

-  Provides  composable  combinations  of  cost  and  value  drivers,  methods, 
probability  distributions,  mission  scenarios,  calibration  data,  outcome 
visualization  capabilities 
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Project  Results  Overview 


*  Successful  project  workshop  in  July 

-  Overall  framework  converged 

*  Thorough  state-of-the-art  review 

*  Several  approaches  being  pursued 

*  Some  initial  useful  models,  tools,  and  case  studies  developed 

-  Total  Ownership  Cost  for  foreseeable  change  cases 

-  Knowledge  Value  Added/Risk/Real  Options;  SHIPMAIN  cases 

-  New  value  based  flexibility  framework 

*  Plans  to  extend,  unify  approaches  and  tools 

-  Methods,  data,  case  studies 

-  Address  unforeseeable  change 

-  Gap  and  opportunity  analyses 

-  Unified  flexibility  analysis  toolset 
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Plans  and  Priorities  for  Phase  2  (March  2011) 


•  Cross-fertilize,  extend  current  valuation  methods  and  tools 

—  Coherent  analytical  framework  for  valuing  flexibility 

-  Calibrate  to  each  others'  data 

•  Assess  improved  methods  and  tools  for  achieving  flexibility 

•  Prototype  unified  flexibility  analysis  toolset 

•  Provide  methods  and  tools  to  Systems  2020  pilots  to  assess  utility, 
obtain  improvement  insights 

-  Tailor  current  methods  and  tools  to  support  upcoming  decision 
situations 

•  Explore  use  of  self-adaptive  project  and  change  traffic  monitoring, 
synthesis  of  recommender  systems  for  adaptation 

-  Management-controllable  cost/schedule  driver  adjustment 

-  Time-determined  development  (DSB,  2007):  add  or  drop  borderline- 
priority  features  to  meet  cost/schedule  targets 

•  Present  results,  obtain  feedback  at  NDIA  SE  conference,  SERC 
Annual  Research  Review 


10/20/2010 


RT-18  Results  and  Plans 
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Deliverables  for  Phase  2  (March  2011) 


•  Prototype  integrated  toolset 

-  Inputs  and  outputs  include  those  needed  for  total  ownership  cost, 
real  options,  knowledge  value  added, ... 

-  Guidance  and  checklists  for  preparing  inputs 

-  Exercise  alternative  approaches  with  cases  from  different  domains 

•  Calibration  of  toolset  to  data  from  different  domains 

-  Modular  munitions 

-  Ship  maintenance 

-  AEGIS 

-  C4ISR 

•  Guidance  and  examples  of  use  in  making  va I ue-of-flexi bi I ity  DoDI 
5000.02  milestone  decisions 

-  Identification  of  when  particular  methods  are  appropriate 

-  Specifying  bounds  on  value  of  flexibility  measure 


10/20/2010 


RT-18  Results  and  Plans 
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Seeking  Input  from  Key 
Constituents 


Conversation  Starters 


•  How  do  you  currently  make  decisions  regarding  flexibility? 

-  How  much  are  you  willing  to  pay  for  added  flexibility? 

-  What  are  the  primary  hurdles  in  valuing  flexibility? 

•  What  information  to  you  need  to  value  flexibility? 

-  Is  the  information  available? 

-  If  not,  what  are  possible  avenues  of  getting  the  information? 

•  How  would  you  use  tools  for  valuing  flexibility? 

-  What  are  the  desired  characteristics  for  such  tools? 

-  Who  would  use  them? 


11/9/2010 


RT-18SERC  ASRR  Review 
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Background  Slides 
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Flexibility  -  Definitions 


•  Flexibility  is  defined  as  the  degree  of  responsiveness  for  any  future  change 
in  a  product  design  [Rajan  et  al.]6 

•  The  ability  of  a  system  to  adapt  to  uncertain  internal  or  external  changes 
affecting  its  value  delivery,  in  a  timely  and  cost-effective  manner.  In  other 
words,  flexibility  is  the  ease  with  which  changes  in  value  delivery  in  a 
system  can  be  addressed.  Here  ease  refers  to  the  cost-effectiveness  of 
addressing  change.  [Nilchiani]8 

•  Distinction  between  ability  to  change  within  a  state  (adaptability)  and 
ability  to  change  from  one  state  to  another  (flexibility)  [Bordoloi  et  al.] 10 

•  The  ability  of  a  system  to  change  on  demand,  incorporating  scalability, 
evolvability,  maintainability,  and  adaptability  [Brown  and  Eremenko]11 

•  Adaptation  is  the  enhancement  or  change  of  a  fielded  system  and  if  such  a 
change  has  a  low  cost-benefit  ratio,  as  defined  by  the  customer  or  market, 
the  system  is  deemed  flexible  [Mark]12 

•  Reconfigurable  system  is  one  in  which  variables  that  can  be  changed  as 
well  as  the  range  of  their  change  are  identified  to  improve  the 
performance  of  the  system  [Ferguson  and  Lewis]13 
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Measures  of  flexibility 


■  Early  design  stage  metric  to  rate  flexibility  of  design  using  flow  analysis  tables 
[Cormier]14 

■  Use  of  entropic  measure  to  measure  routing  flexibility  for  a  job  (part)  [Yao]29 

■  Program  valuation  technique  based  on  Real  Options  Theory  (geometric 
Brownian  motion  used  to  calculate  E[NPV])  [Peoples]15 

■  Development  of  a  measure  of  adaptability  for  products  through  the  use  of  an 
adaptability  factor  representing  the  normalized  savings  achieved  by 
adaptation  versus  dedicated  product  (applicable  to  new  or  existing  designs) 
[Gu  et  al.]18 

■  Information-theoretic  measures  (entropic  measure)  to  measure  routing 
flexibility,  operations  flexibility  and  loading  flexibility  in  a  manufacturing 
system  [Kumar]28 

■  Measure  of  flexibility  based  on  performance  increase  (output)  corresponding 
to  the  required  cost  and  time  to  realize  the  change  [Mark]19 

■  Formal  Methodology  for  the  Evaluation  of  Design  Alternatives  (MEDA), 
rooted  in  classical  utility  theory,  in  order  to  evaluate  and  rank  design 
alternatives  early  on  in  the  design  process  [Thurston]17 
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Incorporating  flexibility  in  systems 


•  Modular  Design 

-  Subordination  of  a  system  to  a  rational  functional  structure  as  an  approach  for  increasing 
general  adaptability  (design  for  unforeseen  events)  [Hashemian]14 

-  General  design  guidelines  based  on  functions  independence  (clustering  and  modularization) 
[Gu  et  al.]18 

—  Modular  design  to  achieve  adaptability  without  change  propagation  throughout  the  entire 
product  [Chmarra  et  al.]20 

-  Clustering  of  components  of  an  adaptable  system  based  on  Design  Structure  Matrices  (DSM) 
[Arts  et  al.]22 

•  Product  Family  Development  /  Platform  Design 

-  Development  of  a  metric  to  assist  with  the  evaluation  of  design  options  early  on  in  the 
design  process  by  rating  the  overall  flexibility  of  the  system  using  flow  analysis  tables 
[Cormier  et  al.]14 

-  Two  stage  optimized  design  process  for  flexible  product  platform  components,  evaluated 
based  on  expected  Net  Present  Value  using  Monte  Carlo  simulation  [Suh]25 

-  Development  of  a  framework  to  increase  the  system's  flexibility  of  fielded  products  by 
evaluating  Optimal  Point  Designs  against  Platform  Based  Derivatives.  Evaluation  is  based  on 
the  performance  gap  of  the  two  designs  [Mark]19 
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Incorporating  flexibility  in  systems 


•  Autonomous  Adaptation 

-  Development  of  an  autonomous  material  flow  system  through  the  use  of  function-oriented 
modularization  and  intelligent  multi-agent-systems  to  enable  the  system  to  react  on  events 
autonomously  [Wilke]30 

-  Three-layer  reference  model  for  self-management  systems  based  of  off  the  three-level  robotic 
architectural  model  Control:  reactive  feedback  control,  Sequencing:  reactive  plan  execution 
and  Deliberation:  planning  developed  by  Gat  [Kramer  et  al.]31 

-  Service-oriented  component  model  to  support  the  construction  and  execution  of  component 
based  applications  that  are  capable  of  autonomously  adapting  at  runtime  due  to  the  dynamic 
availability  of  the  services  provided  by  constituent  components  [Cervantes  et  al]32 

•  Design  Selection 

-  Generalized  purchase  modeling  approach  to  develop  a  Decision  Support  System  (DSS)  based 
on  a  customer  expected  utility  metric  to  support  the  selection  in  product  design  [Besharati  et 

al.]34 

-  Real  Options  concept  to  model  risks  and  delayed  decision  benefits  under  uncertainty  in 
technologies,  etc.  and  development  of  a  quantitative  measure  of  the  value  (performance  and 
cost)  of  different  family  designs  to  select  the  most  appropriate  design  from  all  alternatives 
[Gonzalez-Zugasti  et  al.]35 

-  Staged  deployment  approach  to  introduce  flexibility  into  a  system  using  real  options  (where 
demand  follows  a  geometric  Brownian  motion)  to  match  the  system  evolution  path  to  the 
actual  unfolding  demand  scenario  [De  Week]27 
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Strategies  for  incorporating  flexibility 


*  Delayed  differentiation/extensibility 

-  Flexibility  conservation  principle 

-  Field  programmability 

*  Modularization/reconfigurability 

-  Optimal  decomposition 

-  Interface  standardization 

*  Robust  optimization 

-  Maximizing  range  over  which  value  proposition  remains  feasible 

-  Graceful  degradation 

*  Options  exercise  games  -  intersection  of  real  options  analysis 
with  game  theory 

-  Flexibility  as  a  measure  of  future  options  available 

-  Key  decisions  are  valuing  the  option  and  determining  the  execution 
time 


12/21/09 


67 


